Papers with question answering datasets
Trick Me If You Can: Adversarial Writing of Trivia Challenge Questions (P18-3)
Copied to clipboard
| Challenge: | Existing question answering datasets are imperfect tests that do not expose model limitations. |
| Approach: | They develop an adversarial writing setting where humans interact with trained models and try to break them. |
| Outcome: | The proposed model-driven annotation process systematically stumps automated question answering systems. |
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One? (2022.findings-emnlp)
Copied to clipboard
Xilun Chen, Kushal Lakhotia, Barlas Oguz, Anchit Gupta, Patrick Lewis, Stan Peshterliev, Yashar Mehdad, Sonal Gupta, Wen-tau Yih
| Challenge: | Existing sparse retrievers lack the ability to match salient phrases and rare entities in the query. |
| Approach: | They introduce a dense Lexical Model that can be trained to imitate a sparse one. |
| Outcome: | The proposed model outperforms sparse retrievers on a range of tasks including five question answering datasets and the MS MARCO passage retrieval. |
Towards the First NLP Benchmark for Ladin - an Extremely Low-Resource Language (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are limited in low-resource languages due to lack of labeled training data. |
| Approach: | They propose to use Ladin as a model for sentiment analysis and question answering by incorporating Italian data into machine translation training. |
| Outcome: | The proposed method improves on existing Italian–Ladin translation baselines. |
HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing question answering datasets focus on dealing with homogeneous information, but using homogenous information alone might lead to coverage problems. |
| Approach: | They propose a large-scale question-answering dataset that requires reasoning on heterogeneous information. |
| Outcome: | The proposed model can achieve an EM score of 40% while the existing model is far behind human performance. |
Simple and Effective Unsupervised Redundancy Elimination to Compress Dense Vectors for Passage Retrieval (2021.emnlp-main)
Copied to clipboard
| Challenge: | Dense passage retrieval improves ranking accuracy in open-domain question answering but at the cost of large space and memory requirements. |
| Approach: | They propose a simple unsupervised pipeline that includes principal component analysis (PCA), product quantization, and hybrid search to improve space efficiency. |
| Outcome: | The proposed pipeline achieves good accuracy–space trade-offs, for example, 48 compression with less than 3% drop in top-100 retrieval accuracy on average or 96 compression without drop in space requirements. |
Adapting Entities across Languages and Cultures (2021.findings-emnlp)
Copied to clipboard
| Challenge: | a structured knowledge base adapts named entities using their shared properties. |
| Approach: | They propose automatic methods to adapt named entities using shared properties . they compare them to human adaptations using a new dataset of human adaptation data . |
| Outcome: | The proposed methods compare to human adaptations using a new dataset. |
QAInfomax: Learning Robust Question Answering System by Mutual Information Maximization (D19-1)
Copied to clipboard
| Challenge: | Existing models are not good at distinguishing distractor sentences which look related but do not answer the question. |
| Approach: | They propose a method to regularize question answering models by maximizing mutual information among passages, questions, and answers. |
| Outcome: | The proposed model achieves state-of-the-art on the Adversarial-SQuAD dataset. |
CorefQA: Coreference Resolution as Query-based Span Prediction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing coreference resolution models suffer from mention proposal. |
| Approach: | They propose a query-based span prediction task that can retrieve mentions left out at the mention proposal stage. |
| Outcome: | The proposed model can retrieve mentions left out at the mention proposal stage and improve generalization capability using existing question answering datasets. |